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subsections of the Teacher Rating Questionnaire and the MAT, the 
discr-^pancy scores, and the I.Q. scores. Teacher ratings of 
perfornance vere much more closely related to these measures than 
were their ratings of pupil adjustment and creativity. Despite 
differences among teachers and among classes, as veil as the 
develcpaent in the pupils* characteristics fioa grade 3 to grade 6, 
there vas a reasonable level of agreement among teachers* ratings of 
student performance. The data supports the coDclusiun that teachers* 
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FOREWORD 



Teachers' rating?, in one forn or 
another, help form the basis of many decisions 
made about a pupil’s progress. The relationships 
between such ratings and standardized achievement 
tests will therefore be of interest to educators. 
Although any study in which these relationships 
are examined will be ’'technical," the following 
report attempts to present that information so 
that it j s meaningful to a person who has little 
familiarity with statistics. 
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INTRODUCTION AND REVIEW OF LITI:RATURE 



How valid are teachers' judgements of their papils' ‘ohool 
success? Teachers are asked on many occasions both formal and inforna*^ , 
by principals and parents, to evaluate aiid assess the progress of the 
pupils in their classrooms. Since "achievement’’ within the school sj^te.u 
jTi'dy be interpreted as being clr ;ly related to the interaction between 
teacher and pupil, it could be suggested that the teacher's rating of 
a pupil proi^ides the most appropriate measure of the pupil's aohievemonts. 
However, a pupil's performance on a standardized test of achievement often 
is considered to bo the best indicator of hts achievements* 

What is the relationship between these two measures of achieve- 
ment? V/hen a teacher is asked to nuke a rating or judgement of a 
particular pupil, she will usually rate him relative to the performance 
and behaviour of the other pupils in the class or in the school. Con- 
sequently, an average pupil in a class consisting primarily of below 

average pupils nmy be rated as "outstandj.rg, " because relative to the 

other pupils in the class he is an above average student. On ihe other 
hand, if this saiie pupil were in a class of primarily above average 
p>upils, his performance might be rated as "below average" relative to 
that of his classmates. Ch’* a standardized test, however, each pupil's 
scores are coriparod to a comrion referent, i.c. the norms develoj^d for 
that particular test. Such norms are based on the perfo 'ranee of a large 

number of pupils in a large murjber of different classrooms. Docs the fact 

that the teacher uses her own class as a "measuring stick" against vrU.ch 



to assess a particular pupil result in a rating which is inconsistent 
with that pupil's performance on a standardized test? 

/'I though there has been some work done to investigate the 
relationship between teachers^ ratings and standardized tost measures, 
it has not been particularly extensive or detailed, 

Finley (1966) compared pupil performance on three subtests, 
la]:guage arts, arithmetic reasoning and reading comprehension, of 
three different achievement tests with teacher ratings obtained when 
tie pupils v;ere in grade th.'^=e and grade five. The teachers w^ere 
asked to indicate wheiher, in general, each piipil in their class was 
performing at, belov;, or above the grade leve] , She found some disagreeme; 
fiiTjong the differerr: standardized tests, i,e. pupils obtained higher 
scones on the California Achievement Test (CAT) than on either the 
i'otropolitan Achievement Tests (MAT) or the Iowa Test of Basic Skills 
(ITBS), as well as some disagreement betw^een the standardized tests and 
the teacher ratings, ITie results; of the series of comparisons between 
each of the standardized tests and the teacher ratings are as follows: 

(1) California Achievement Test - In both grade 
+}iree and grade five, teacher rating scores 
were lower than scores obtained on all 
^,hree subtests of the CAT, 

(2) Metro r olitan -chievenent Tests ; In grade 
three, the teacher ratings w'ere the same as 
tb^ scores on two subtests, language arts 
aiid reading comprehension, but lower than 

M'll arithTiOtic subtest scores. In grade five, 
the results wore exactly the opposite, i,e. 
the teacher rating scores w^ere lover than the 
MAT scores on the langu ^gc arts and reading 
comprehension subtosts and the sane as the 
MAT arithm.etic subt^'st scores* 

1 California Achievement Test, Metropolitan Achievers lit Tests and th'*^ 

Iowa Test of Basic Skills, 
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(3) lova Test of Basic Skills : With one exception, 

the results of the comparison between grade 
three and grade five teacher ratings and the 
ITBS were identical to those discussed above 
for the liAT. 

These differences in the extent of the agreement between 
teachers' ratings and achievement scores, depending upon both the 
particular achievement test employed and the grade of the pupil, suggest 
that caution is required in interpreting any such future cor.parisons. 

It should not be concluded that teachers' ratings of school performance, 
because they do not agree with achievement test scores, are poor estimates 
of the pupils' accomplishments. In fact, the differences may arise in 
part because the teachers' ratings allow for the inclusion of aspects of 
performance that ura obvious to the teacher, but arc not measired by 
the pencil and paper achievement tests. These differc^'ces may also arise 
in part because the achievement tests are based on nationally-standardized 
norms whereas the teachers' ratings are largely based on the norms that 
exist for that one class. 

In another study (Ebbesen, 1968), teachers v;ere asked near the 
end of the school y<^ar to put their kindergarten pupils into rank ord‘-'r 
in terms of how far they thought each pupil would go in school. At the 
end of the first, second and third grades respectively, these pupils were 
again u! ac.ed in rarik order; however, :n these instances, the ranking was 
done on the basis of the MAT scores obtained for that year. To measure 
the degree of correspondence between the teacher ratings and tlie MAT 
scores, three correlation coefficients were calculated, one for each grade 
Remembering that 1he maximiun positive value of a correlation coefficient 
is +1.00, representing a perfect correspondence between two s<^ts of rcor'^s 
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the correlations obtained were: 0^60 in grade one; 0.54 in grade i.v;o; 

0.52 in grade tliroe, T}ie sirnilai'ity of the three coefficients iudicaten 
that a knowledge oi the teacher's ranking of the pupils in kindergarten 
provides us good a prediction of their Mj\f score in grade one as it does 
in grade three. Alt. hough hbbesen felt that the correlations were qu: ^ 
low and indicated a fairly Darge amount of disagreement between teacher 
and achievement test rankings, it must be remembered that there v;ere few 
opportunities available for kindergarten teachers to observe their pupils 
performiTig the sorts of tasks that in later grades were to form the basis 
for the acliicvement test e^^aluations. In addition, the rankings in 
kindergarten v/ere made on the basts of only one criterion, a prediction 
of futxire educational success. In light of these two points, 'ebbrse]Js 
findings seeri very encouraging. 

In another study of kindergarten pupils (Meyers, Attw-11 eu.l 
Orpet, 19c3), teachers v:ere asked to rate the overt behaviour r>:hibi^c.: 
by their pupils during the administration of a series cf achie/errnt 
tests. Thn behaviours rated included: amount of motor activity, rr^n lal 

dexterity, attention and effort displayea. Vdien the pupils v:-^re 'n 
grade five, they wore given the California Achievement T<?’sl. r- ruV, s 

of this test were corrt ared witli the achievement test results ’u.d b*‘'lre/icur 
ratings obtained . heri the pupils were in kl i*idc’rgarten. It was fcuri i ‘ 
one of thr> behaviour ratings, attention, was almost as good a i r* dlctor 
of the grarb^ five achieverm rit test scores as aiiV of \no ki: d«rg-ad ‘:>rj 
•ichievemer.t tests, i.<^. the maximum correlatiosi betw^ n any of th^ kinbr- 
garten subsets and the grrade five scores W':l 0.50 w!,' roas th.'^ correlr;t ii. n 
between attejition and tb<e grade five nchlevov.ent test scores was 0.4fn 
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Although t}>? three studies just discussed have at ^ re. ‘.ch-- i i.t'-’ 
problorrf in sor.ewhat different v:ays using different st‘:^ndard.i :v,d 
neat 1<?sts and puj-dle at a n^unber of grade levels, they liu ]U' i«„ ror/- 
evidence for a relationship beWeen teacher ratings and starri-a' : 'i 

t' st rrjeasurrs. 

It y-is suggested in the introduction that teachers v'-r/ ^ 
tlieir jud(y'r',ents on a relative as opposed to an absolute scale, 
their judgen;ents of pupil perforiaance are related to the e-xp^.r.^ j jons 
an(i standards they have established for their ovn class , such ■md .rdc, 
of course, vai^ii.g from class to cla^^ . The follov/ing studios j.rovi'to 
sore on: ] ort for inis suggesticn. Although dubbard and s}ou* (1953) 
gave .heir te=vcl;ers criteria for judging I.ds. they fou’id t :,at 
jU'^.geruoat s V'-re tdf'hly influr^jcedi by the r articular ra.nf/'- of inlel i ;■ a 
t-''rst scoies of the jUjils in tlieir class. i)..a fih'bing v.as cosU'iri'.'d by 
vU'-'-x-ar/bru' ( 1953 ), who fouiui ^hai ^ 'iclau' jud^fr.r-rits of ‘'ax-'* id ^o 
vanicli a :u:il van \;or>:ing up to }:is cajacif/ wore iiif luraac': 'i ly ^ h.e 
norr'.s thai fa*- * -yaha- r bar' -~‘S tablisiied for his cl'uun The *^x: r/'cha' ra 

obs''-rV'"'U Vaal alM.eugii th‘*> a ■ dr: rs v;orc '^bl'*’ to raak "laeir ruii'lr 

according to ^‘n*'ir ability or l-'vel of achi '*>ver/'>ru s , th<*' nua.erical v *-^U‘*r 
tliat t ],•■■;/ gav< as -a: 1 i na1 -n' of 1. 9. 1‘rrfpuc at ly uer‘' conai r" onl ly loo 
iiigh or too lev for ony given class. 

}ial'err/ui and Katbs(l9a^) cor/ucted -‘x r,or' dir‘'ct ir.v* stigation 
c-f 1 critr-ria use! by t' acl.^-rs iii s-l ‘"■c 1 iiyj higli- or lo' ‘--ae].i''-vi].g 
nrils. .■.acj'i of a n^ucbei' of rixt >*-grad«-> t^-acta-^r' uas giv-.a a, ficMt iour. 
list of j utils ani trnur gredf;^ loint avrragrs, }',ach t' acla r v:as givrn 
a different list^ so-:.'^ lists liad a vid^' rat.g'*' of gi'a b' roir.t av^rag- s, 
vliile oth-'^rs bjad a narj'uva-r rai.ff, restricted to thos'^ scer^es falling 
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tov;ard the centre of the distribution of possible scores. They were 
asked to select from the list the pupils that they thought would benefit 
from either enrichment or remedial classes. The results showed that the 
teachers selected the same number of pupils from their list irrespective 
of the range of grade point averages. Tnis would seem to inaicate that 
they were selecting pupils according to their standing relative to the 
rest of the class, not according to the pupil's absolute score. 

On the basis of the above findings, it may be stated that 
teachers' estimates of their pupils' abilities and accomplishments 
differ somiewhat from the estimates obtained using standardised tests. 

The sample of pupils included in each of these studies was usually 
relatively small, i.e. 100 or less, and the teachers asked to rate 
their pupils in terms oi' only one attribute or characteristic. Sine- 
data collected as a part of the longitudinal Study of Achiever.ent 
initiated b;,^ the Research Department in 1960-1961 iixludod scorr^s for 
a laige nwnber of pup>ils on the following measures: 

(1) Teacher Rating Questionnaires adrdnistered 
in grade three and grade six; 

{2) Metropolitan Achievem.ent Test adriinistered 
in grade three; 

(3) Otis Quick-Scoring Mejital Ability Test 
(r.ew edition — Alpha Short Form) 
administered in grade tuo; 

it v'ar possible to examine in som*e detail the relatioijship betvreen 
teachers' ratings of a pupil's school success and standard! red test 
measures of hds school achievement. 




10 



- 7 - 



PROCEDURE 



Pupil Population 



The data on which this paper is based were collected from 520 



pupils who were a part of the basic population of 8,695 in the longitudinal 
Study of Achievement. There are two reasons for the small nuiriber of pupils 
included in this study relative to the larger basic population: 

(1) Although during the first five years of the Study 
data were collected from the entire sample of pupils 
available, in grade six, the teacher ratings were 
obtained only for those pupils whose mothers had 
been interviewed during Lhe previous year, i.e., 1967, 
when the pupils were in grade five^, 

( 2 ) Of those 721 pupils, I.Q, scores, MAT scores, and grade 
three teacher ratings were available for 520 pupils; 
grade six teacher ratings were available for only 429 
of these 520 pupils, as some had left the system or 
wore absent during testing. 



Short Form) is a standardized test of intelligence which v;as adninist' r-: 1 
during grade tv;o. 



subsections, i.e. Adjustment, Ferfcrmance, Creativity and lYediction 
of School Success, each containing a number of separate but related items. 
The teacher is asked to give each pupil a rating of either 0, 2, 4, 6 
or 8, depending uion tlr? extent to which the pupil displays t’ie cliaract'U'- 
istic in. question. A bri^f d'^scriptive statement accomi: arjies <~ach j orriM^ 
rati rig -iiA p rovider the teaclier wi + h a gjiiieline for dc terminirig vhicli 

2 For a dircussior. and more detailed infcrr.at ion concerr-ing thv^ 

dal a, ^ he reader is referred to S c ! 1 0 0 1 A c h i v e me r it : A 1 r 1 i r. i r~. a 2 v I 0 o >: 
Q at the Effects of the Home . Research Depart me lit, 1970. 



Descri.ption of the Measures Used 



The Otis Qijick-Scoring Mental Ability Test (new edition -- Alnlia 



"^be Toachter Rating 'Ques t ionnaires , for grade three and grade 



six were develcj^d by the Research Department, They co: sist of fcair 





- 8 - 



value is jaost appropriate for any given pupil. The following item from 

the grade six Adjustment subsection is given as an illustrative exajnple: 

General Adjustment Evaluation 

Considering all aspects of the child’s adjust- 
ment to the classroom environment^ evaluate 
his position. 

Rate 0 — Quite out-of-place. 

Rate 2 — Seems uncomfortable. 

Rate 4 — Adequately adjusted. 

Rate 6 I^fe.kes consistent, conscientious 

efforts to improve himself in 
relation to his school world. 

Rate 8 — Makes an effort to positively 
influence his classroom world. 

The complete questionnaires for both grade three aj:d grade six 
as well ns a brief history/ detailing the development of the Te=ich‘::^r Rating 
Questionnaire may be found in Appendix A. It should be noted some 

modifications were made cn individual items in the quostionnnir? ..-etween 
grades three and six. These changes were based on experiences v/ith the 
quest icnnai7'e in grsde three. 

The standardized achievement test employeci was the li e trcroli tas 
A c h i e V r me n t Tests, : - 1 e me n t a ry Ra 1. 1 e ry . admiidstered in g^^ade threo, 
areas covered by this test series include word knov7jr*^ige, word vii scr ir.l r.n- 
tion, reading, spelling, aiuthr.etic computation, aritbir.etic problem 
soDving, langnjage usage, punctuation and capitali tation. riach pupil 
r-:'ceivcd a score representing his perfornance on each of the above sub- 
tests; these subtest scores are not combined during the scovir.g of tb-^ 
test to give a total score r':T'rrs'^i.ti)ig an individual pupil 
•Vvel. ror tbm:' jurposcs of this s"ud;/, hovver, average yf-.T scor-'^s r-‘ 
c^ilculat^^i for raci; pupil. 
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One iTiOans of evaluating a pupil's school progress is by looking 
at his actual achievements relative to his predicted achievements. A 
Discrepancy Score was devised to measure the extent to which each pupil's 
actual achievements corresponded to what it night be expected his achieve- 
ments should be. On the basis of this score, the pupils v;ere categorized 
as either under-, average-, or over-achievers. A brief digression is 
necessary at this point to explain the rationale and teennique for deter- 
irJ-ning t. is score. 

In order to laake precise predictions about any pupil school 
achievements, accurate quantitative measures of his actual and potential 
achievements are required. Typically, it is expected that pupils of 
higher levels of measured intelligence (l.Q.) v;ill do better in school 
than their colleag^ies with lower l.Q. scores. The^-efore, in some 
instances l.Q. scores cind achievement test scores have been directly 
compared, i.e. pupils whose achievemient test score was less than their 
l.Q. score were identified as under-achievers, end pupils whose achicvG~ 
ment test score exceeded their l.Q. score were labelled as over-achievers, 
Ho'wever, serious problems associated with the use of this simple comp's risen 
procedure led Thorndike (1963) to develop a more refined technique for 
identifying over- and under-achievers. Ihorndike's m"’thod significantly 
reduces the effects of one of the sources of error associated with any 
kind of testing, that of random or chance error. For example, w^henove' 
a pupil is given a test of any sort, his final score will likely be 
influenced by a ijiarr:ber of l actors in addition to the knowlrdge that he 
is asked to cormundcate in answering the test questions. For example, 
if he were not feeling well on the day of the test, hd s score mjdght be 
lower U.-in it wo ild le on another occasion. If he correctly guessed sore' 

o 

ERIC 

MfflifflMliiTifaii 
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of th-; ans.cers, his score would be higher than it should be. If, when 
taking an I.Q. test a pupil correctly guessed some of the answervs and 
received a high score, and took a standardized achievement test on a 
day when he was not feeling well and received a low score, he would 
probably be categorized as an undnr-achiever, whei’eas, if he had taken 
both tests under idea], conditions, he might have been categorized as 
an average-uchiever, Thorndike's (1963) statistical technique for 
determining a predicted acldevement score reduces by a considerable 
amount the cliances of such erroneous classifications. 

A second feature of Thorndike's technique is that it allows 
the establisyjr.ent of several groups which have different discrepancy 
scores, i.e. under- or over-achievers, but all of whic?i have the same 
average I.Q. score. Consequently, in each group there are equal numbers 
of pupils with high, average or low I.Q. scores, i.e, each I.Q. score 
is equally represented in each of the groups. 

Description of Thorndike Statistical Technique for Deteminin/:; 
a Predicted Achievement Score 

In order to determine an individual's discrepancy score, a 
knowledge of both ‘lis predicted and actual achievement scores is n^c<*^'rsa 
Ihorndike's forr.ula for deterrrining the predicted achiev.?r.ent score , ijt 
this case, t?ie predicted Metropolitan Achi eve:r.ent Test score, is as 
follows : 



Predicted MAT score = Individual's I.Q. score X 

correlation t )tween th ■ set of 
I.Q. and actual MAT scores for 
the group3 



Kxpess^d verbally, this forrmjla means that each juril's pr^'^aictei aciur* 
ment ccor*:' is d^:^termineci by multiplying his I.Q. scor«-« by corr“0 a i c 



3 

O 
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it should noted that the successful application of this t-^ciiniju? 
r-' n-iroe that I.Q. and achievement test scores to available for ^ 
faii'ly large group of lupils. A class of thirty is sugg'^stei -iS 
miidrnjm. 
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obtained between the I.Q. scores and actual achieveirient scores for the 
group being studied. The reader will remember that a correlation is 
always obtained bet\-:een two sets of scores ^ and is a measure of the 
extent to which, on the average, an individual's score on one test 
can be predicted from his score on another test. This correlation 
coefficient functions as a correction factor for random error (as dis- 
cussed above) that may exist in the data. When the predicted MAT 
scores have been obtained, the difference between the predicted and 
actual MAT scores can be calculated for each pupil and a decision ir.aae 
as to whether each pupil falls into the category of under-, average-, 
or over-achiever. If the pupil's predicted fiAT score (what he should 
be getting, given his I.Q.) is higher than his actual MAT score by at 
least a certain amount, he is designated an under-achiever. A rovrr:ni 
of tiiis relationship identifies the over-achiever. 

Calculation of the Discrepancy Scores for the Pupils in t he 
Study of Achievement 

Before calculating predicted MAT scores, the actual M.'T scores 
and the I.Q. scores were converted to standard scores. Inis is a r/'ans 
of making scores obtained by different testing procedures r/ithcracically 
co:\] arabl^a Issentially, the process involves taking each indivi mmol's 
score on a test, wsub-*. racting it from, the group average, a i dividing t}>-^ 
result by the standard deviation^. Using these standard score forms 
of the actual MAT scores and the I.Q. scores, predicted MVT scores v;ere 
obtained using the method described in the preceding rectlon. differ-n 

between the predicted and actual MAT scores was obtained 'ovi the 

4 l:o standard d^viulion is an index of the ar.ou* t of mi at ion in t}? 
tes\ scores. By dividing each person's differer.c*: .zC K'o by this ini*?:*:, 
each of t}>a origir.al sets of test scares is transform,'^ i so that all "i.- 
sets have equal ?:r.oun‘' s of vari?ition in them and also leave the ':v-r 

score, thereby making the data easi^^r to work wit),. t runs lorm.a t ic rt 

no v:ay char.gos the relationships among tho scores within ic.divi iual 

Q or betw*^^'.:: tho two sets of scores. 

ERIC 
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sorted into five categories in terms of the following criteria: 

(1) If the difference between predicted and 
actual MAT scores was greater than +1,00 the 
pupils were classified as either extreme 
over- or under-achievers, according to the 
direction of the difference; 

(2) If the difference was between ± 0,50 and 

- 0,99# the pupils were classified as either 
over- or under-achievers; 

(3) If the difference between predicted and 
actual MAT scores was less th^n ^ 0,49# the 
pupils \.ere classified as average^achievei'S. 

The number of pupils falling into each of 
these groups is shown in Table 1 , These 
numbers are also expressed as a j)ercentage of 
the total population of 520. 



TABLE 1 

HUMBER AND PER CENT OF GRADS THREE PUPILS IN EACH DISCRSPAiXY SCOR^: 
GROUP# ALONG WITH THEIR AVERAGE I.Q, ;JJD MAT SCORES 



Achievement Group 


No. 


Per Cent 


Mean 


lie an 


MAT Score 


0 f Total 


I.Q. 


Verbal 


Arithmetic 


1 - Extreme Under- 
Achiever 


55 


10.57 


104.38 


2C.85 


16.14 


2 - Under-Achiever 


83 


15.96 


104.76 


23.98 


19.04 


3 - Average-Achiever 


245 


47.11 


106.57 


29.29 


22.14 


4 - Over-Achiever 


91 


17.50 


105.91 


32.86 


25.46 


5 - Ex^ reno 0-/er- 
Aohiever 


46 


8.84 


103.33 


35.35 


27. 74 




IB 



It should be noted that although the average I.Q. score for 



each group is approximately the same, i.e. about 105, the average scores 
on the combined verbal or the combined arithmetic subsecti<^:is of the 
increas e as the actual MAT score increasingly exceeds the predicted MAT 
score, i.e. as a progression is made from Group 1 through to Group 5. 
Thus there are now available for study five groups of pupils who are 
virtually identical in one respect, i.e. average I.Q, score, but who 
differ in another respect, their discrepancy score, or the extent to 
which their actual achievement scores correspond to their predicted 
achievement scores. 



17 




- 14 - 



ORGANIZATION OF THE RESULTS 



As outlined in the previous section, the following data were 
available for the pupils included in this study: 

(1) I.Q. score; 

(2) Teacher Ratings in grade three; 

(3) Teacher Ratings in grade six; 

(c) Metropolitan Achieverient Test scores 
obtained in grade three; 

Discrepancy Score, which could range 
from 1 to 5j where a score of 1 
designated an extreme under-achievcr 
and a score of 5 designated an extreme 
over-achiever. 



These data were used in the following comparisons to examine 



in detail the different measures used to assess pupil progress in schco 

(1) The teacher ratings in both grade three and grade 
six were examined to see to what extent teachers 
avail themselves of the full range of possible 
scc: 7 es when evaluating pupils. Do teachers tend 
not- to use extreme scores and show a preference 
foi* 'Vaverage'' ratings, or dc they tend to avoid 
’’average” ratings? To what extent are teacher 
rating scores distributed in the same manner as 
scc«:7es on standarcii^^^d achievement tests? 



(2) Avc-;7age fiAT scores were compared with teacher 

rat ings made in grade three to examine the degree 
of ngreemient between teacher ratings and standardized 
tei IS completed in the sar:e year. 

(3) Avc:rrige M\T scores were compared with teacher 

ratiLiigs in grade six to give some ii;diCcation 

of '.he accuracy; of long-range predictions hascd 
on a knowledge of the average MAT scores. 




18 



- 15 - 



(4) A comparison of the grade three and grade six 
teacher ratings was made to determine the extent 
to which ratings made in later grades, i.e. 
grade six, could be predicted on the basis of 
earlier ones, 5. ,e. grade three. 

(5) Teacher ratings for both grade three and grade six 
\4ere compared with the discrepancy scores to 
establish the extent to which teachers distinguished 
between over-, average-, and under-achievers in 
their ratings on pupils' adjustment, performance, 
creativity, and prediction of school success. 

(6) I.Q. scores were comipared with average scores. 
I.Q. scores were compared with teacher rating 
scores in order to determine whether the relation- 
ship between I.Q. and teacher ratings \:as as strong 
as the relationship between I.Q. and ^iAT scores. 

(?) I-!AT scores, I.Q. scores, and discrepancy scores 
were combined in a muitiple regression analysis 
in order to determine the optimal combination of 
these three different measures for the prediction 
of teacher ratings. 
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RESULTS 



The Distribution of Teacher Rating Scores, 

MAT Scores and Discrepancy Scores 

The first step in the analysis was to determine whether each 
of the achievement measures was normally distributed across the range of 
possible scores. Each of the distributions w^as divided at three mathematically 
equivalent points into five sections, and the percentage of the total niamber 
of scores falling into each section was calculated. These data are presented 
graphically in Figure 1. It is evident that all three measures are dis- 
tributed in a similar manner, i.e. the largest percentage of the scores 
fall in the middle of the distribution, with approximately equal per- 
centages falling at either end. Although this general pattern is exemT-lified 
by each of the measures, the distribution for both the MAT scorr-s and iho 
discrepancy scores is moi-e accentuated than that for the teacher rating (TR) 
scores, i.e. a larger percentage of the MAT and discrepancy scores fall 
at the Centre of the distribution (/vA and 47^ respectively) as compnrr^i 
’With the TR scores (32%) » The teachers did avail themselves of th^- full 
range of possiblr^ ratings when assessing th«ir pupils and in *idd’ Lion., 
they cended to use the ’’average'^ .^ating just about as often as .he ratir.gs 
at either extreme. 

Vhen TR scores were compared with the standardised test jcor^'s, 

TiOre pupils were rated as average on the basis of the MAT results, (i.‘n 
44;t) thai. on the basis of the TR results (i.^. }2%) * 0n.<^ ir.i licat ion cf 
this finding is that teachers cat* diseririlnate bn tier ano? g ttmir 
th»an can standardized tests. It r.ust be rer.em.lv^red, }.oaa^v‘*’r, tha^ 

Teaciier Rating Quest ion.naires tap a wider rangr of classrcor* 1* havlcurs 



20 



Pi » CliLNT OF THE TOTAL SCORES 



- 17 - 



than do standardized tests and so allovr the teachers to form a more 
distinctive description of their pupils. 




X- AVtPiGE L'V c-;-;.Lo 3 

CFATE C lE/XHiP PATING:; 
(TOTAL SCORES) 

■ 

■ - average mat sccfe 

a - Dls:f>JANCY SCGFLS 




REGION or THE DIS7R1BDTI0V IN GTANDARB TEVUTrON UNITS 



Fig. 1. Id strllut ioiiS of tliroc achif'verxnt iv'anm 
r.enn M\T rcor-'s nni discrepmcy scorr.^. 



■: teacher rntif, 
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T:ie R e lationship Between Teacher Rating Scores ar/3 Average M/iT Scores 

The scores for ®ach suhsection of each of the grade three and 
the grade six teacher ratings were correlated with the average MAT 
scores (see Table 2). 



TAHT.K 2 

CORRKbATTO::S OF mat scores with teacher rating subtest scores 





Adjustirit^nt 


Performance 


Creativity 


Predicti on 


Total 


Grade 3 


r 


.50 


,69 


.50 


, 6? 


,66 




.25 


,47 


.25 


.45 


,44 


Grade 6 


r 


• 42 


.62 


.45 


,65 


.61 


2 

r 


,18 


.38 


.21 


,42 


.37 



" To obtain the per cent of variation accounted for by "r”, 
r^ by 100, 

Tnsp^ctioii of Table 2 revealed that in boih grade three and gmd^ nix 
tnere vms a strorger relationship between the average M/\T scores 'ir.'l 1j. 
scores on the perfoiTOnce (r = ,69 and ,62 for grades three and six 
resr>:-ct ively) and r^rcdiction [r " ,67; ,65) sections thari belv-'r^en 
average MAT scores ar/i the adjust r.^^nt (r ~ ,50; ,42) and creativity 
'r “ .50; ..' 45 ) sections of the Teacher Rating Quest iennairo, Since the 
M'\T tajis I'l'inarily acad*:^riic perforrar.ee, it- \ras not surprising M.at 
there should be a strorger relationsIMp wit}i tV.ose s--'ctio:'S of th--' 

ratir-gs r<"lated to acaderic j'erfon'.ancc as cor.pared witn Erse- 
s' ctions eoi/rorr.r i with other aspcc'^s of th*:- j urilr’ classrcon h<~havic'; 



o 
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It should be noted that while the absolute values of the JIAT-TR correlation 
ranged from ^42 to .69, the two sections most relevant to standardized 
test scores, performance and prediction, maintained values in excess of 
,60 over the three yocar interval, A correlation of .60 iniicates that 
36^ of the variability can be explained and, while modest, is quite respect- 
able when maintained ovei* time. 

One further point must be made regarding the relationship 
between the teacher ratings and the avera^ l-tAT scores. Although, as 
described above, the same pattern of results was obtained for both grade 
three and grade six, the values of the correlations obtained between the 
grade six teacher ratings and the mean JiAT scores were slightly lov;er 
than those obtained with the grade three teacher ratings. There are at 
least two factors v:hich could account for the smiall decrease in tiie value 
of these correlation coefficients; 

(a) the correlation coefficients calculated 
using the” grade six data w’ere based on a 
smaller mimber of ]:apils, i.e. 429, as 
compared with the 520 students for whomi 
grade throe teacher ratings wmu'e avail- 
able; and 

(b) the lapse of three years between the 
administration of the and the graae 
tliree teacher ratings and the adriinistra- 
t ioii of the grade six Teacher P.ating 
Questionnaire w’ould likely account for 
some decrease in iliese correlation co- 
r'fficients. Ii.decd, the decrease is 
rorutriiably smiall given a tir.e interval 

of throe years. 

To s’OTJ'.ari th^re is a, rolatio!:slnp botwf;^<^n th^ averu;^ '‘AT 

subs'^clicris of tlm^ Teaciior Rating 
stror.gost for toac*v>r ratings 
1 id''orr,'H'.ce and nr^aMciion. sai - 

va s aclmi n i s t re li . r > Vi - r , ^ 1 , 



scores and the scores for the varicus 
QUest * oMwiire. "“iiis rolaticrsVjip vos 
cbtaiT.rd in gr-a ie « i } ar* i cul'-.r 1 y 
s^-c^-ions), srcao y^ .'r that t}>'- M-M 
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results ir.dicutcd that, the accuracy of long-range predictions (i.c. grade 
six) would be slightly less than that of short tern predictions. One 
eould hypothesiae that one possible reason for the discrepancy betv;een 
average M.AT scores and teacher ratings in both grades tnree and six is 
that the class norns on which the teacher ratings are based V'ould usually 
include a more restricted range of pupil abilities than the standardized 
norms developed for the iUT and based on a naticn-v;ide sample of pupils. 



er|c 



Prediction of Cfrade Six Teacher Ratings fron Grade Three Teacher Hatini;;s 

lo vhat extent Ju evaluations of pupils made in cne grade agree 
v.’ith evaluations made in later grade? Evaluations m.3de of a pupil 
school performance and adjustment tend to follow that rjupil throughout 
his school career in some form of perjTianent recoi'd, Vdicn a pupil eii or:? 
a new grade, the new teacher will likely consult ^.hr- v;ith the 

intention of gaining background information and infornation ^hout 
schccl success. The ratings are, however, made n^ar the end of the school 
year: they are r.-wie by different teachers ctod tlie clasrr^a'es of j-ut iI 

in gr.ade six are unlikely to have ber>n classma.'-vm ir'j grade tiuvo. 

'ibw^reforo sorw' disagrer'r/a.l betwcf^n the ratings of ora^ y^^ar and rxnxt 
could be exr^'ct'd. Although the teacher ratings obtaij\^-d for th'^ Itudg 
of Aeniovcmeiw. do not aii^^ar otj t }-upil’s i rchool r^o-'^rd, *:wy 

wer».^ ceijsidered to he cinilar lO descrlritivp ccr_m’n'*'G a ^e’-ariw r migh* 
includ*:' on t?v 0. f.Fh 

Corr^d atio.ns a’«^u'e obt'tin^d b*d,w- er. t !'■> cfiri r-s; ondii.g sul c*: c t ions 
of grad<' thre^ and grade six t‘'ach‘‘r m'Wings av.d ar*' ju'^ s^aw d in 
T'-.bl*^ 3* Ii'- addition to *h^' ccrrelat i cti corf fioi ^ n* s , ■ iw? ]‘-r c^nt cf 
varialillfy in <bita that is ^'xpl'iiu'd Vy •'ach -.n ff i ei^u.t is rr.ov::,. 

On'arlo fciiool tycor't 
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These values (i.e. ) represent the accuracy of the prediction that can 

be made of the grade six teacher ratings from a knov;ledge of the grade 
three ratings. 

TABLE 3 



CCRRELATIOMS BEr,'EEN THE COmSPONDIliG SUBSECTIONS OF THE 
GRADE THREE GRADE SIX TEACHER RATING QUESTIOrJNAIRES 



Subsections Correlated 


r 


2 

r 


Adjustment 3 x Adjustment 6 


.54 


.29 


PerforiTiance 3 x Performance 6 


.57 


.32 


Creativity 3 x Creativity 6 


.41 


.17 


Prediction 3 x Prediction 6 


.53 


.27 


Total 3 X Total 6 


.60 


.3b 



(N = 429) 

It Is obvious that the best prediction (36/f) could be made when the 



total scores on the questionnaii es were employed. The porformanc'^, adjust- 
ment and prediction subsections allowed 32, 29 and 27 per cent accuracy 
respectively, while crecativity allowed only 17 per cent, IhiL rcderate 
agreement betv;een scores for grade three and scores for grade six does 
not suggest either that some teacher ratings w^ere ^'wrong," or that teacher 
ratings are therefore poor instruments for pupil r?valuatien. Rather, ihr.y 
do imply that ) redict ions^ from or.e grude to cinother must be r/nie v.uth 
.mome cautioni, lofferences In the two s»^ts of ratii.gs ceuld be due to at 
le^st two factors : 

(a) changes in ttie pupils' tnhaviour during the 
three yerirs intervenirg between ihr- two s*^ts 
of ratings; ciijd/cr 

L.b) th^ fact that the ratings in grad‘d tlir^'-e aid 
gred<^ six wore ccrpletcd by two di f f^:r*>nt. S‘^ts 
of teachers who nay have ap^lird iiffer' r.L 
cri * . 
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It is likely thai the adequacy of the frames of refer^rino +}-;■=/, 
teachers have at hand varies with the t^^pe of behaviour to be rated, Kore 
clearly defined standards are available for judging academic perfox iiance 
than creativity, due to the nature of the school curriculum. This would 
account for the lov,est correlations being obtained for ratings of 
creativity and highest for performance. The values of the correlations 
serve as an indication of the extent to which teachers are in agreement 
as to the criteria or standards to be used in rating pupil behaviours. 

Comparison of Teacher Rating Scores ar.d Discrepancy Scores 

For each of the TR subsections, n^an scores were calculated 
for each of the five discrepancy score achievement groups. These cal- 
culations, coi;ducted separately for boys arid girls, are iresenled in 
Table 4 and illustrated graphically in Figures 2A and 213, 

An examination of the mean scores for both boys and girls in 
be oh grades showed that for all of the TR subsections, the teachers rated 
over-achievers, l.e. Groups 4 and 5, higher than they rated average- 
achievers, i.e. Group 3* In addition, both over- and average-achievers 
were rated higher than uiider-achi evens, i.e. Groups 1 and 2. This 
relationship was most pronounce! for the performance and prediction sub- 
socticc.s as is r>vi<icnc*:'d by sU ‘‘r sloj-:^s of the d-pl cting 

the rolationsbiip botwcon those two subscctic’is --.nd tip''' discrotanop scor^'^s, 
as seer, in Figures 2A ar.d 2B for gradr-s thr***'' '\rA six r-'-srectiv'*!;/. 

Vfneth<'‘r or not a pu]dl is aij over-, ov und-r-aeijiev^r is :\cst 

strongly r''"flcct<^d i: t'^'achors' ratings of his ac’-. pcrformar.c^' and 

to e consid'^ rabl;,' 2<^srer d'^gre*'^ isi ^ ''s of hir p' rrral classrrom 
U'havicur. it will b« r''mwmb‘-'r»' d t:rw this p'ff'-'rn of rrsulis, 
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TABLE 4 

t-EAii tt:acki:r rating subsection scores for each of th- 

FIVi:: DISCREPANCY SCORi; ACHIEVPU'ENT 
CROUPS, FOR BOYS AND GIRLS, R£:SPECTIVE'LY 



Teacher Patiiigs 




Achievement 


Groups 




1 


2 


3 


4 


5 


Grade 3 


Adjustment - Male 


3.95 


4.46 


4.65 


5.59 


5.87 


- Female 


4.13 


/. . 57 


5.55 


5.85 


6.70 


Pe- r f 0 r rna nee - Ma 1 e 


2.80 


3.71 


4.47. 


5.45 


-6.83 


- Female 


2.84 


3.59 


4.38 


5.45 


5.83 


Creativity - Kale 


3.35 


3.54 


4.36 


5.42 


5 . 30 


- Female 


3.18 


3.59 


4.21 


4.55 


4.64 


Prediction - Male 


2.35 


3.33 


4.39 


5.42 


5.57 


- Female 


2.18 


3.08 


4.08 


4.75 


5.30 


Grade 6 


/VJjustment - Male 


3.40 


4.79 


4. 6l 


5. CO 


5.69 


>“ Femalp 


4.03 


5.06 


5.05 


5.51 


6.00 


Performance - Kale 


2.91 


4.00 


4.20 


5.03 


5.89 


~ Female 


3.36 


3.90 


4. 36 


5.^8 


5.78 


C;'eativity - Male 


3.41 


4.10 


4.17 


5.15 


5.50 


- Female 


3.23 


3.59 


3.88 


4.7m 


t t'.C’ 


Prediction - Kale 


2 . 34 


3.75 


4.27 


5.45 


6. 1 1 


- Female 


2.39 


3. ^'4 


3. 90 


5.12 


5. 
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• • Boys 

^ O Girls 






0^ ^ — i — i — i — ^ 

IN:;ER- over- 

ACHIVVES ACHIEVER 



1 I I I I 

0 ' 2 3 < 5 



ACHIOEMIRT OROJPS 



Fig, 2A. Moan scores for (‘ach subscc:ion of iho grode three 
Teacher Rating Questionnaire, calculated separately for Toys ar.d 
girls in each of the five achiever, ent groups. 
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# • Hoys 

O O 




6 r CBEAiivm 




2 - 



1 • 

0 ' V — J — 5 —^ — i 

UN!)EB- QVKR- 

ACHIEVEP ACHTLVER 




A CHUVEXENT GROUPS 



Fig. 2P. Kean scores for each subsection of the grade six 
Teacher Rating Qaiestionnaire, calculated separately for boys ani 
girls in each of the five achie/ererjt groups. 
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as discussed previously, characterized the relationship between the TR 
subsection scores and the average WT scores. 

Although there was little change in the mean ratings between 
grades, there were differences within each discrepant achiev^'ment group 
between mean scores obtained for girls and those obtained for boys. The 
mean ratings v;ere higher for girls than for boys on the adjustment sub- 
section, and highei' for boys than for girls on the creativity and prediction 
subsections* Boys and girls were not rated differently on the performance 
subsection. Since boys were rated as likely to go further in school, 
although their acac(?mic performance was rated being similar to that of 
the girls, it seems evident that factors other than academic perforriance 
alone were influenciiig the teachers’ predictions of how far their pupils 
would go in school. The most probable factor is a still '-existing orj'iGct ation 
in our society that boys, as a group, will obtain more education than 
girls. Given equal intellectual abilities, boys will more likualy 
encouraged to continue their school ir.g, and in turn, it is likely that 
teachers, on the avarage, resjo:.d‘ d accordingly in their ratings. 

The h i tionshiu Between I.Q. Scores and Avera ge 

MAT Scores and Tn ^ ■ , bor Rating Scores 

To jeovide a measure of the degree of relationshil' b^'tween 
I.Q. ecorcw and each of the acjnevm”:^nt measures, i.?. M-.T scores a]‘.i 
teacher ratings, co3'relation coeffici'-'iits v.-err- calculated (see Table 5)* 
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TABJ^ 5 

CORRELATIONS OE I.Q. WITH MEAN MAT SCORES AND 
TEACHER RATING SHBTEST SCORES 







MAT 


Adjustment 


Performance 


Creativity 


Prediction 


Total 


Grade 

r 


j. 


.54 


.25 


.40 


.37 


.^3 


.41 


2 

r 




.29 


.06 


.16 


.14 


.23 


.17 


Grade 

r 


6 




.25 


.39 


.32 


.45 


.39 


2 

r 




— 


,06 


.15 


.10 


„20 


.15 



Tho correlation obtained between the 1,Q. scores and the xr, 
MAT scores was .54. The correlations between I.Q. and each of the four 
TR subsections and the total score in both grades three and six rangerl 
from .25 to . 48 ; therefore, the perforriance subsection of the Teacher 
Rating Questionnaires was a better predictor of the average ?<AT scores 
than the I.Q. scores (see Table 2). Although it is evident that a 
knowledge of a pupil's I.Q. score does not enable one to make a very 
accurate prediction of either his average MAT score or his TR su’ section 
scores, as noted in a previous paper^, the pattern of these results is 
encouraging as 

’^..they seem to indicate that the teacher 
uses the I.Q. information in those Instances 
where it likely has most relevance, i.e. 
predicting how far the child will go in school 
and is least influenced by it in her rating of 
whether the child presents a discipline problem 

. . .continued 



6 Eor a more detailed discussion of the importance of I.Q. as a predictor 
of a child's school perfonr.ance, the reader is referred to School Achieve 
A Preliminary I>ook at the Effects o f the Hone , Research Department, 197C. 
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in the classroom. Children with lover I.Q. 
scores generally were no more likely to be rated 
as being discipline problems than children with 
higher I.Q. scores.'^ 

(Crawford & Eason, 1970, pp. 18-19) 

As was pointed out in School Achievement: A Preliminary Look 

at the Perfects of the Home , although I.Q. was a better predictor of both 
the MAT scores and the TR scores than either socio-economic status or a 
sample of home environment factors, it could explain a iriaximium of only 
30^ of the variability in these achievement measures. There are undoubt- 
edly factors other than the pupil's measured intelligence that strongly 
influence his school performiance , whether that performance is measured 
by a standardized test or by leachers' ratings. 

MAT Scores. I.Q. Scores and Discrepancy Score s 
as Combined Predictors of Teacher Ratinf^s 

The correlations obtained between the teacher ratings and each 
of the standardized measures included in this study, i.e. I.Q, scores, MAT 
scores and discrepancy scores, indicated that some relationship exists 
betwr^n those measures. The question arose, was there a vaay of comparijjg 
all three standard! /:ed measures simultaneously to determine if there aas 
a comb: nation of the measures which would increase the accuracy of predict "ng 
the teacher ratings as compared with that which could be obtained using 
any one r.^asure alone? One way of accomplishing this goal was to utilize 
t}]^’ statistical technique of nilliple regressiori dnalysis in vlucl. the 
teac}:<:>r ratings were designated as the criterion \nariablcs and I.Q., 

M\T scores and discrepancy scores were th*^ predictor variables. 

.Miltiple regression analysis is a complex form; of correlaiicn 
analysis that pro}'ort ions variablrr or: t!;e basis of th^ir correlaticn 
with tl;e criteric'i variJ^-ble in such a way as to provide th'^ l^st r^ssibl*-, 
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i.<^. maxiiniajn, pradiction of criterion variable. An optimal propor^' 1 ot>- 
ing of the piedictor variables is accomplished by assigning v^eights to 
all the variables so thct the most efficient predictor I'cceives the 
largest weight. Ihltiple regression analysis selects from among the 
predictor variables only those variables that add to the acc-a'acy of the 
preaiction. hventually, a point will be reached where the addition of 
further variables does not increase the accuracy of the prediction. At 
+his point, a numerical /slue is obtained indicating how much of the 
variability in the data has been accounted for by the selected predictor 
var^iablr>s. Thus multiple regression analysis selects the minimum numl^'r 
of variables that give a maximum ariount of prediction, hlien £ll the 
predictor 'variables originally entered into the regression are good 
predictors, the minimum numiber of variables selected as contributing 
to the prediction of the criterion variables will be the same as tlic 
total number of predictor variables originally entered, 

Inter;Tetation of a multiple regression analysis is similar 
to the inter]>ret at ion of a sim.ple ccrrelrition . The value of R obhai:>:ri 
by multiple regression ar.alysis is sinpiv the correlation betwerr, tie' 
criterion variable and the s^uri of the weighted predictor v.'sriaM* :’. 
is ar. indication o"' the amount of variation in the data that is acceun'e-; 
for by the selected set of predictor variables, 

bsing the scores for each subsect io:; and the to-al score ol’ 
ti*/-' teacher ratings as tli^ critcrioi* variables, and t]*^ averag-- MU 
scor-'^s, I.T'. scares and discrcpaiicy scores as pr^^cictor varrhol'S, 
two sets of ;v.iltiT'"o regression armbiys^s were con.iucted, for ‘■‘ach 
of grade *.hr> (? an i grade six. ih^ ia:'suMs of tJm.s*' ar/j]yr<''S a}*> sarc rir*’ 
in Tablr^ 6, 




33 



- 30 - 



TABLE 6 

VALUES OBTAINED FOR EXPRESSING THl'; D1FFERF,NT MULTIPLE REGRESSION 
EQUATIONS f'OR EACH OF THF^ TEACHER RATING SUBSECTIONS IN BOTH GRADE 

■]'hr;'e and graj)e six WKf:N achif:vement groups, average mat scores 

AND I.Q. lrt:RE THE PREDICTOR VARIABLES 



Criterioii 


R 




Predictor 


Variables* 




Regression 

Constant 


Achievement 

Group 


I.Q. 


M/iT 


Grade 3 














Pd justirient 


.4976 


.2476 


.0371 


— 


.4673 


2.56>08 


Perforric:i:ice 


.6899 


.4759 


.1061 


.1086 




-11.2330 


Creativity 


. 5' 46 


. 2648 


— 


.1347 


.4473 


-4.6489 


Prediction 


.6842 


.4682 


.0614 


.2071 


.5083 


-4.5362 


TOTAI. 


.6658 


.4433 


.0483 


.0964 


. 5727 


-17.6299 


Grade 6 














Ad justnent 


.4264 


.1818 


.1439 


. 1 353 


.2370 


2.8092 


Perfornar.ee 


6233 


.3885 


— 


.1279 


.5217 


-10.1022 


Cr-:>ati vity 


.4630 


.2190 


.0525 


.1584 


.3322 


-3.114" 


Prediction 


. 6620 


.4383 


— 


.1735 


.5395 


-4.8525 


TOTAL 


.6103 


.3724 


.0829 


.1605 


.4540 


-15.3324 



^ TIk v'llucs listed for each of the three predictor variahler e.r-' tiie 
beta weights obtained in the regression analysis. 
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The accuracy of prediction obtained for each of the teacher 
rating subsections for grades three and six respectively v/as; adjustment, 
25% and 185C; performance, lS% and creativity, 26^ and 22^; prediction 

of school success, Ul% and IJS\ total score, 44^ and 37?. 

A comparison of these values of obtained from the multiple 
regression anfilyses, with the corresponding values of r^ for the correla- 
tions betvjeen each of the grade three and grade six teacher ratings and 
the average JiAT scores (see Table 2), indicated that very little vas 
added to the predictability of teacher ratings uhen all three standardized 
measures were combined. 

Further confirmation of tie importance of the ro^3 of the pupils’ 
v;ork performance to the teachers^ ratings com.es from an examination c the 

7 

beta weights of the multiple regression analyses. In each of the t€:n 
analyses, the highest weight was assigned to the average MIT scores, meaning 
that these scores contributed the most to the prediction ^f each of thm^ 

TR subsection scores. The contributiai of the I.Q, and discrepancy scores 
vas ajjproximiately one-half or less of the contribution of MvT scores. 

7 These are the weightings which, as described on page 29, permit 
maximum predict ability through the optimal proportioning of the 
predictor variables. 
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SUMMARY AND imiCATIONS 



In this report, teacher ratings were compared with standardized 
measures of achievement to determine the extent of their relationship. 

The question of interest was, does a teacher’s rating of a pupil in her 
class provide the same information about his level of achievement as 
scores he obtains on a standardized test, the Mietropolitar Achievem.ent 
Test? 

Quite bluntly, the answer to this question is no I Although 
there is some relationship between these two measures, the degree of 
relationship depends cn those areas in which we ask the teacher to inake 
ratings! An examination of the data obtained in both grade thiree and 
grade six revealed a consistent pattern in tlie relationship between 
various subsections of the Teacher Rating Questionnaire and the M\T, the 
discrepancy scores and I.Q. scores. Tc^.chers' ratings of pupils' perform- 
ance was more closely related to these measures than were their ratings 
of pupils’ adjustment and creativity. 

Similarly, teachers in grade three and grade six wer^ greater 
agreement about their ratings of performance and prediction tl;an heir 
ratings of classroom behaviour and creativity- Furthermore, the teach‘::‘rs' 
ratings of perfoiTiance provided a better prediction of the achievei'enl 
test scores than did the I.Q. test scores* 

It appears then that teachers’ ratings are good iredictors i-- 
certain areas. In spite of differences among !:eachers and among classes, 
as well as tho development in the pupils’ characteristics from grade three to 
grai<^ six, th-^re was a reasonable F"vel of egr'^ement riror.g t};*^ t^ach^-rs 
in tlieir ratings of students perfo^rkance. Teacher differences in 
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dGiin:tions of vhat constituted a particu?.ar type of behaviour were more 
apparent in the ratings of classroom behaviour^ and, especially, of 
creativity. A lack of some clear, generally available, criteria applicable 
to these areas is evident. Teachers' judgements of "high” and "low" 
achievement were consistent with the achievement test results. This 
suggests that the teachers do indeed use classroom performance as a 
criterion in raking their ratings much more than any knowledge of a 
recorded I.Q. score. This is supported by the regression analyses. 

In conclusion, the data suggest that teachers' ratings of academic 
performance are an important and dependable adjunct to the use of standard- 
ized achievement tests. 
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STUDY OF ACHIEVHffiNT 
STAGE V 



AIMNISTRATION BOOKLET 

TEACHERS' PARTING QUESTICITOAIRE 
(CKADE 3) 



INSTRUCTIONS 



1. Please read each questior. carefully. 

2. Decide from your own knowledge the ratings for each child. 
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ADJUSTMENT TO SCHOOL PROaiAMi-l£ 



1 * Discipline 

Displays behavio\ir that you i the teacher, co: ider appropriate for your 
classroom. 

Rate 0: Constant discipline problem; behaviour al ways inappropriate . 

2 : Frequent discipline problem; behaviour often inapprc}pria te« 

4: Occasional discipline problem; exercises some self opntrol . 

6 : Very seldom causes discipline problems, exercises s elf control 

most of the time . 

8 : Never causes discipline problems, behaviour always appropriate . 



2. Acceptance of Routines 

Accepts responsibility in connection with classroom v/ork, seatvork, rcutin^: 
roles and directions. 

Rate 0: Never accepts r CvS pons i bill ty; nee ds constant help and attention 

from ceacher . 

2 : Seldom accepts responsibility; has to be coaxed , inconsistent 

in his response to routines. 

4: Frequently accepts respons"^ bility; tr ies to look after his 

tasks . 

6 : Regularly accepts responsibility; looks after his tasks almost 

always . 

85 Consistently accepts responsibility; looks after his tasks 
successfully all the tine . 
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3. Acceptance of Goals 

Shows d esire to make positive contributions towards classroom activities, i.e. 
answers questions readily, moves in gym willingly, sings in nusic periods, 
talks during discussions. 

Rate 0: Shows no interest in the activities, makes contribution. 

2 : Shows limited interest in a few activities. 

4; Responsive towards numerous activities, able to contribute 
sometimes . 

6: Shows interest in a great number of activities, contributes 

often . 

8: Is interested in all activities and contributes whenever 

possible. 



4. Ability to Get Along 

Interacts with most of his classmates in a satisfactory manner. 

Rate 0; Unable to get along in classroom, (or in schoolyard), always 
quarrelsome in social contacts. 

2: BYeouently quarrelsome , or limits social contacts to one or 

two chosen friends . 

4: Gets along with most pupils , and regularly participates in 

group activities. 

6: Often shows leadership ability in group activities, and is 

popular with most classmates. 

8; Consi^ tently shows leadership ability in social contacts, and 
is trusted by other children. 
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WORK PHRFOm-IANCE 



5* Attention and Work Completion 



Has good attention span, is able to attend to teacher and assignments without 
constant encouragement. 

Rate 0; Extremely short attention span, easily distracted ^ seldom i f 
ever, finished assignments. 

2: Short attention span, easily distracted, gets v/ork done 

occasionally . 

4: Able to listen for the duration of the lesson, usually gets 

his work done* 

6: Above average attention span, gets his work done regularly . 

8; Superior attention span, will work at any task as long as 
necessary, vill it is completed. 



6. Reading 

Reads with compr^^hension and fluency, conveys meaning to listener. 

Rate 0; Reads with little or no comprehension, mostly word by word * 
without much meaning. 

2: Reads with word recognition and comprehension at bottom level 

of class. 

4: Reads with comprehension and fluency, conveys meaning at 

middle level of class* 

6: Reads with word recognition and comprehension at top level 

of class, 

8; Superior reader, able to comprehend most material encountered, 
e.g., magazines and books at higher grade levels. 
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7, Language, Self E?cpresslo n 

Can tell or write ’'stories”; expresses self clearly. 

Rate 0; — Occasionally attempts to tell or write a ’’story”; 

— ’’Stories” consist of one or two sentences; 

* Sentences may be completely xmrolated, 

2; — Frequently attempts to tell, or write, a ’’story”; 

— ’’Stories” have many irrelevant ideas , 

4: — Regularly attempts to tell or write a ’’story”; 

— Few, if any, irrelevant ideas, 

6; — Consistently attempts to tell or write ’’stories”; 
— Few, if any, irrelevant ideas; 

— Occasional use of complex sentences . 



8; — Tells or writes coherent ’’stories”; 

— No irrelevant ideas, use of complex and compound sentences; 

— Unusually good comnana of language. 



8, Accuracy and Qu^it.y of Work 

Can usually do work correctly. ' 

Rate 0; Consistently makes errors in copying and seldom, if ever, 
does assignments the right way. 

2; Inconsistent both in accuracy of copying, and in doing assign- 
ments , 

4: Does work the right way, but needs supervision, 

6: Does work the right way and seldom makes errors, 

8; Wo/k always accurate with quality beyond requirements. 




U 



9. I^themtical Ability 



- 41 - 



Shows understanding of laathamatical concepts and opsrations, can colve 
p. oblens . 

Rate 0: Very limite d ability to understand mathematical concepts and 

operations, cannot solve problems, 

2; Mathematical understanding end problem solving ability is 
at lower level of class . 

4? Usually able to understand mathematical concepts and operations 
when presented by teacher* 

6; Mathematical understanding and problem solving ability Is at 
upper level of class * 

8; Superior mathematical ability, - immediately mderstands 
mathematical ideas presented by teacher. 
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CRMTIVE THINKING 



FCR YOUR GUIDANCE - the following meanings are intended when these words sre used. 

Intuition ; — Immediate insight; 

— Immediate apprehension by the mind without reasoning. 

Divergent : — Capable of going in different directions; 

— Differ from the usual. 

Inventiveness ; — Ability to devise, to originate. 

Imagination ; — Mental faculty of forming images of external objects not 
present to the senses. 



10. Imagination and Inventiveness 

Regardless of academic achievement , he may :e considered imaginative and 
inventive. 



Rate 0: Never shows imagination or inventiveness . 

2- Rarely shows imagination or inventiveness. 

4: Occasionally shows imagination or inventiveness . 

6: Frequently shows Imagination or inventiveness . 

8;» Regularly shows Imagination or inventiveness. 
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11. Use of out>of -school experien ce s in class 
Draws on background experiences. 

Rate 0: Shows no background experiences, reports no InTormatlon 

pertaining to the world about him. 

2: Shows a few background experiences, reports some information 

pertaining to the world awut him. 

4j Reasonably veil informed . 

6: As a result of his background experiences, he is often able 

to contribute new information . 

8: As a result of his backgi'ound experiences, regularly displays 

a wealth of knowledge. High degree of sensitivity 
to the world aroiLnd him. 



12. Creativity 

Shows an urge to explore and create; is intuitive. 

Rate 0: Always placid , never shows signs of curiosity, no capacity to 

be "disturbed." 

2 : Rarely shows curiosity or the desire to explore. 

4: Occasionally displays signs of divergent thinki ng. 

6: Frequently displays signs of divergent thinking , has a great 

urge to explore. 

8: Regularly displays signs of divergent thlnking < possesses the 

rare gift of Immediate insight . 
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SCHOOL ABILITY 



13* To provide your estimate of this child’s ability , try to predict how far 
you think he will go (ignore financial ability of parents). 

Rate 0; Will have difficulty completing grade eight. 

2 : Will not complete high school. 

4; Will complete high school. 

6: Will go to university. 

8: Will go beyond a B.A, 
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STUDY OF ACHIEVEMENT 



STAGE VIII 



ADMINISTRATION BOOKLET 
TEACHERS » RATING QUESTIOiraAIRE 

(Grade 6) 

May, 1968 



Inst p-ictions 



1. Please read each question carefully. 

2. Decide from your own knowledge the ratings 
for each pupil. 

3. Use the special pencil to nark the rating 
on the pupil's I.B.M. card. (Please check 
that the column used matches the item being 
marked.) One mark and one mark only per 
column. As there are only twelve questions, 
you only have to nark the first four coluirjis. 
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ADJUSTMENT 



1 , Discipline 



Displays behaviour that you, the teacher, consider 
appropriate , for your classroom. 

Rate 0 - Conctant discipline problem; behaviour 
a lways inappropriat e. 

Rate 2 - Frequent discipline problem; behaviour 
often Inappropriate , 

Rate 4 - Occasional discipline problem; exercises 
some self control % 

Rate 6 - Very seldom causes discipline problems; 

exercises self control most of the time . 

Rate 8 - Never causes discipline problems; 
behaviour always appropriate. 



2, Ability to Get Aloj^ 



Interacts with most of bis classmates in a satisfactory 
manner. 

Rate 0 - Never able to get along in classroom 
(or in schoolyard). 

Rate 2 - Frequently quarrelsome, or limits social 
contactr to one or two chosen friends. 

Rate 4 - Gets along with most_pupils , and regularly 
participates in group activities. 

Rate 6 ~ Often shows leadership ability in group 
activities, and is popular with most 
classmates. 

Rate 8 - Consistently shows leadership ability In 
social contacts, and is trusted by other 
children. 
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3 • Acc eptance of Goals 

Contributes to classroom activities, i,e., answers questions 
readily, talks during discussion, makes active contribution 
to class projects. 

Rate 0 *- Shows no interest; makes no contribution. 

Rate 2 - Shows limited interest in a few activities. 

Rate 4 *“ General interest in classroom activities; 
contributes occasionally . 

Rate 6 - Wide variety of interests; contributes 
regularly . 

Rate 3 - Participates actively and enthusiastically 
in all activities; contributes more than 
do most other pupils. 



4. General Adjustment Evaluatio n 



Considering all aspects of the child's adjustment to the 
classroom environment, evaluate his position. 

Rate 0 - Quite out-of -place. 

Rate 2 - Seems uncomfortable. 

Rate 4 ~ Adequately adjusted. 

Rate 6 - Makes consistent, conscientious efforts 
to improve himself in relation to his 
school world. 

P^ate 8 - Makes an effort to positively influence 
his classroom world. 



?! 



o 

ERIC 



- 48 - 



PERFORMANCE 



5. Reading 



Reads with comprehension and fluency; conveys meaning to 
listeners. 

Rate 0 - Reads with little or no ccinprehension, 

mostly word by word , without much meaning. 

Rate 2 - Reads with word recognition and comprehen- 
sion at bottom level of class. 

Rate 4 - Reads with comprehension and fluency; 

conveys meaning at middle level of class. 

Rate 6 - Reads with word recognition and compre- 
hension at top level of class. 

Rate 8 - Superior reader, able to comprehend most 
material encountered, e.g., magazines and 
books at higher grade levels. 



6. Mathematical Ability 



Shows understanding of mathematical concepts and operations; 
can solve problems. 

Rate 0 - Very limited ability to understand methematical 
concepts and operations; cannot solve problems. 

Rate 2 - Mathematical understanding and problem 

solving ability is at lover level of class . 

Rate 4 - Usually able to understand mathematical con- 
cepts and operations when presented by teacher. 

Rate 6 - Mathematical understanding and problem solving 
ability is at upper level of class . 

Rate 8 - Superior mathematical ability; immediately 
understands ria^liematical ideas presented \y 
teacher. 
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7. Language 



Extent of vocabulary; correct grajnmatical usage of English; 
ability to express self clearly, (Both oral and written.) 

Hate 0 - Generally very poor command of the 
language, frequently misunderstood, 
or cannot express self due to 
inadequate language. 

Rate 2 - Language poor; on occasion is mis- 
understood. 

Rate 4 - Command of language adequate; child 
still makes some grammatical errors. 

Rate 6 - Good command of language; no gramiTiatical 
mistakes. 

Rate 8 - Extremely articulate for child this age; 

superior vocabulary, clear expression of 
ideas. 



8. Us e of Out-H^f School Experiences in Class 



Draws on background experiences, reading. 

Rate 0 - Shows no background experiences; reports no 
information pertaining to the world :-bout him 

Rate 2 - Shows a few background experiences; 
reports some information. 

Rate 4 - Reasonably well informed . 

Rate 6 - As a result of his background experiences, 
he is often able to contribute new 
information . 

Rate 8 - As a result of his background experiences, 
regivlarly displays a wealth of knowledge. 
High degree of sensitivity to the world 
around him. 
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9. General Performanc*3 Level 



The quality of work; diligence in performing it. 

Rate 0 - Makes many errors ; doesnH concentrate; 
seems uninterested in improvement . 

Rate 2 - Makes an effort to concentrate, still 
has difficulty with work; quite a few 
errors . 

Rate 4 - Listens; performs as required t relatively 
neat. 

Rate 6 - Somewhat above average; diligent; few 
errors. 

Rate 8 - Far above average ; diligent; produces 
extremely accurate work . 



CREATIVE THINKING 



For Your Guldsmoe — the following jneanings are intended 
when these words are used: 



Intuition immediate insightj 

- immediate apprehension by the mind 
without reasoning. 

Divergent - capable of going in different 
directions; 

- differ from the usual. 

Inventiveness - ability to devise, or originate. 

Imagination - mental faculty of forming images 
of external objects not present 
to the senses. 



10, Imagination and Inventiveness 

Regardless of academic acMevement, he may be considered 
imaginative and inventive. 

Rate 0 - Never shows imagination or inventive- 
ness. 

Rate 2 - Rarely shows imagination or inventive- 
ness. 

Rate 4 - Occasionally shows imagination or 
inventiveness. 

Rate 6 - Frequently shows imagination or 
inventiveness. 

Pate 8 - Re gularly shows imagination or inventive- 
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11* Creativity 



Shows an urge to explore and create; Is Intuitive. 

Rate 0 - Always placid , never shows signs of 

curiosity; no capacity to be ^’disturbed." 

Rate 2 - Rarely shows curiosity or the desire to 
explore* 

Rate 4 ~ Occasionally displays signs of divergent 
thinking * 

Rate 6 - Frequently displays signs of divergent 
t hinking ; has a great urge to explore. 

Rate 8 - Regularly displays signs of divergent 
thinking ! possesses the rare gift of 
iiDinediate insight. 
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ABILITY 

12. School Ability 

To provide your estimate of this child's ability « try 
to predict how far you think he will go (ignore financial 
ability of parents). 

Rate 0 - Vill have difficulty completing 
Grade Eight. 

Rate 2 - Will not complete high school. 

Rate 4 - Will complete high school, 

Rr,te 6 - Will go to university. 

Rate 8 - Will go beyond a B,A. 
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A Brief Chronology of the 
Development of the Teacher Rating Questionnair es 



When the Study of Achievement vfas initiated, it was believed 
that the teacher assessment of her pupils was one important indication 
of achievement that must be included. In consultation with the Kinder- 
garten Department, five areas of importance in terms of the kindergarten 
programme were identified: language, mental, social, emotional and 

g 

physical development. Mrs, Helen Gaston, who vas then a kindergarten 
teacher, made a major contribution in the development of the first 
form of the rating scale (Stage II), She designated 40 behaviourally 
defined situations for which the teacher was to provide a rating of each 
pupil. Pilot vjork showed that the teachers preferred a five point rating 
scale as follows: 0, 2, 4> 0, C. On the basis of her exj)erience, Mrs. 

Gaston developed descriptive statements that defined (a) typical or 
average behaviours, to be rated as 4, (b) extreme behaviours to be rated 
as 0 or B depending upon the direction of the behaviour displayed, and 
(c) two interiTiediate categories to be rated as 2 or 6, 

The next year, it was apparent that the rating scale for grade 
one (Stage ill) required modifications because some behaviours considf^red 
to be typical in grade one differed from those considered to be typical 
in kindergarten. The number of items included in this scale was slightly 
reduced (i,e, 33 items as compared to 40 items) and a more elaborate 
behavioural description provided for each item. Miss Elinor Gullette> a 
consultant, did the work on Stage III. 

In this same year, the Kindergarten Questionnaire was revised by 
Mjs. Gaston so that the instructions were clearer. It was made available to 
kindergarten teachers as a device to be used at the end of the year to assess 




o Mjr3. Gs?tor. is currently Kindergarten Supervisor for the Foard of Education 
for the Borough of Etobicoke. 
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the a.jhievorient of thr?ir students. In this revised fono it i/as called 
the "June Questionnaire" and contained 39 items. An additional, modifica- 
tion was made by Mrs. Gaston so that a similar form dealing vit^\ thm 
same topics could be used by teachers early in the school year to -‘nsider 
school progress; this is referred to as the "Fall Questionnaire, " (24 
items). Both questionnaires are available to kindergarten teachers from 
the Kindergarten Department. 

In grade two {Stage IV) a further modification, carried out by 
a cons^iltant, Miss Florence Roliff, reduced the rating scale to 27 items. 
Although there was some variation in content, a serious attempt was made to 
define similar behavioural situations for all three grades. I^ach year, 
comments by teachers provided assistance in making modifications and in 
directing the Department to attempt a shorter, more streamlined question- 
naire. 

For grade three (Stage V) Vxs, Szabo, a vice-principal at that 
time, now a principal, assisted in the questionnaire modifications. As 
a result of previous experience, discussions, and primarily her suggestions, 
modifications were undertaken which resulted In the abbreviated form 
analyzed in this report. This questionnaire now had 13 items which provided 
ratings in the areas ^f Adjustment to School Frograrmo, Work PerforrmiiCe , 
Creativity and Prediction of Schoo] Success, fo furtlmr major rovisior.3 
w^ere mride to this questionnaire although some of the items were modified 
and one item from the section on creativity was dropped in grade four 
(Stage Vl). In grade six (Stage VIII), the questionnaire was identical 
to the one used in grade four. In this form, the questioi;naire seemed so 
successful that it was used as one of the iristmr.ents in tlio Study of A'^w 
Ciuiadians. For this jiurpose it was considered necessary to drop the two 

well as the item on mathematics since in some 






items on creativity ns 
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instances the ratings were being completed by teachers v;ho did not havc- 
the opportunity to assess the behaviours mentioned in these three it^ms. 

The most noteworthy difference between the form used from 
grade three onwards, and the form used from kindergarten to grade two 
is that in the lating scales for the early grades the teachers were 
asked to rate each child in specific situations, e.g., can follow some 
directions in games, shows evidence of eye-hand coordination when building 
with blocks, has ability to control temper. In each of these situations, 
specific behavioural examples were provided for each rating level. In 
the later grades, the situations rated were much more general in nature, 
e.g., can accept responsibility in the class, gets along with most of 
his classmates, can read fluently etc. The typical behaviour for the 
g rade was used as the referent for each of these situations, 

The Teacher Rating ^Questionnaire is now available to bo used 
at any grade level from kindergarten to high school; siirillar mioans nnd 
standard devia*;xons have been found for these questionnaires when used 
at different grade levels. 
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